Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[basicprofiles] Fix StateFilterProfile to use linked Item system unit #18144

Merged
merged 28 commits into from
Feb 12, 2025

Conversation

andrewfg
Copy link
Contributor

@andrewfg andrewfg commented Jan 20, 2025

Resolves #18122

There is a bug whereby the calculations over a StateFilterProfile set of time series state values are based on the Unit of the first member in the set of values. There are two problems: a) the order of the state values is not relevant, and b) the Unit of the incoming state values is not definitive to the Unit of the expected result. Therefore these calculations do produce variable and unexpected results.

This PR fixes the bug by normalising the calculations so that they are all based on the Unit of the target Item to which the StateFilterProfile is linked.

Furthermore this PR allows conversion between non- inverted and inverted Units so invertible conversions (e.g. Kelvin <=> Mirek) are supported.

And finally it also provides better support for cases where the binding may provide state updates in different mixed formats e.g. a mixture of QuantityType with different base units, UndefType, and various States that can yield a DecimalType value.

Signed-off-by: Andrew Fiddian-Green [email protected]

@andrewfg andrewfg added the enhancement An enhancement or new feature for an existing add-on label Jan 20, 2025
@andrewfg andrewfg self-assigned this Jan 20, 2025
@andrewfg andrewfg requested a review from J-N-K as a code owner January 20, 2025 18:33
@andrewfg andrewfg marked this pull request as draft January 20, 2025 18:33
Signed-off-by: Andrew Fiddian-Green <[email protected]>
@andrewfg
Copy link
Contributor Author

@jimtng @mherwege for info..

@andrewfg andrewfg changed the title [basicprofiles] Improve StateFilterProfile unit based calculations [basicprofiles] Normalise StateFilterProfile calculations to use the linked Item's Unit Jan 21, 2025
@andrewfg andrewfg changed the title [basicprofiles] Normalise StateFilterProfile calculations to use the linked Item's Unit [basicprofiles] Normalise StateFilterProfile calculations to use linked Item's Unit Jan 21, 2025
Signed-off-by: Andrew Fiddian-Green <[email protected]>
Signed-off-by: Andrew Fiddian-Green <[email protected]>
Signed-off-by: Andrew Fiddian-Green <[email protected]>
@andrewfg andrewfg added the awaiting other PR Depends on another PR label Jan 23, 2025
@andrewfg andrewfg requested a review from jimtng January 23, 2025 18:38
@andrewfg andrewfg marked this pull request as ready for review January 23, 2025 18:38
Signed-off-by: Andrew Fiddian-Green <[email protected]>
@andrewfg andrewfg changed the title [basicprofiles] Normalise StateFilterProfile calculations to use linked Item's Unit [basicprofiles] StateFilterProfile calculations use linked Item's Unit Jan 23, 2025
@andrewfg andrewfg changed the title [basicprofiles] StateFilterProfile calculations use linked Item's Unit [basicprofiles] StateFilterProfile calculations use linked Item SystemUnit Jan 23, 2025
Signed-off-by: Andrew Fiddian-Green <[email protected]>
Signed-off-by: Andrew Fiddian-Green <[email protected]>
Signed-off-by: Andrew Fiddian-Green <[email protected]>
@andrewfg andrewfg changed the title [basicprofiles] StateFilterProfile calculations use linked Item SystemUnit [basicprofiles] StateFilterProfile use linked Item SystemUnit Jan 24, 2025
@andrewfg andrewfg changed the title [basicprofiles] StateFilterProfile use linked Item SystemUnit [basicprofiles] StateFilterProfile uses linked Item SystemUnit Jan 24, 2025
Signed-off-by: Andrew Fiddian-Green <[email protected]>
@andrewfg andrewfg changed the title [basicprofiles] StateFilterProfile uses linked Item system unit [basicprofiles] StateFilterProfile now uses linked Item system unit Feb 3, 2025
Signed-off-by: Andrew Fiddian-Green <[email protected]>
Signed-off-by: Andrew Fiddian-Green <[email protected]>
Signed-off-by: Andrew Fiddian-Green <[email protected]>
@andrewfg andrewfg changed the title [basicprofiles] StateFilterProfile now uses linked Item system unit [basicprofiles] Fix: StateFilterProfile to use linked Item system unit Feb 8, 2025
@andrewfg andrewfg changed the title [basicprofiles] Fix: StateFilterProfile to use linked Item system unit [basicprofiles] Fix StateFilterProfile to use linked Item system unit Feb 8, 2025
Signed-off-by: Andrew Fiddian-Green <[email protected]>
Signed-off-by: Andrew Fiddian-Green <[email protected]>
@andrewfg andrewfg removed the awaiting other PR Depends on another PR label Feb 9, 2025
@andrewfg
Copy link
Contributor Author

andrewfg commented Feb 9, 2025

@jimtng good news that all your PRs have all now been merged; which means that this one is also now ready to go: => could you please have a look at it?

Copy link
Contributor

@jimtng jimtng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice refactorings!

One thing I'm wondering about is how would it handle the scenario when linked to an item with unit, but the condition is OtherItem1 > OtherItem2 (i.e. nothing to do with the linked item)?

@@ -603,7 +635,7 @@ public int getWindowSize() {
// the previous state is kept in the acceptedState variable
return 0;
}
return windowSize.orElse(DEFAULT_WINDOW_SIZE);
return windowSize.isPresent() ? windowSize.get() : DEFAULT_WINDOW_SIZE;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It eliminates the yellow squiggly underline compiler warning in Eclipse IDE :)

@andrewfg
Copy link
Contributor Author

andrewfg commented Feb 9, 2025

how would it handle the scenario when linked to an item with unit, but the condition is OtherItem1 > OtherItem2

Good question. If I recall correctly I think the code converts 'OtherItem' to the string representation of its State, and then the string is handled in the same way as one hard coded in the condition LHS / RHS. So if 'OtherItem' was a QuantityType we would attempt to convert it to the systemUnit. And in the case 'OtherItem1 < OtherItem2' .. if either or both would not convert to systemUnit we would have an error -- which I guess means that the input State is rejected??

PS in any case I will add some JUnit tests.. => EDIT: I see that there are actually already many such tests .. and they all still pass.

@andrewfg

This comment was marked as resolved.

Signed-off-by: Andrew Fiddian-Green <[email protected]>
Copy link
Contributor

@jimtng jimtng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@lsiepel lsiepel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM

@lsiepel lsiepel merged commit d0bac82 into openhab:main Feb 12, 2025
2 checks passed
@lsiepel lsiepel added this to the 5.0 milestone Feb 12, 2025
@andrewfg andrewfg deleted the statefilter-functions branch February 13, 2025 09:55
@florian-h05
Copy link
Contributor

@andrewfg This change broke a comparison between a MQTT number channel with unit kWh and the comparison string < 50, I needed to change this to < 50 kWh.

The logging was

// before:
ant 11:55:37.211 | DEBUG | org.openhab.transform.basicprofiles.internal.profiles.StateFilterProfile | Evaluating Condition(''  LT '50' ) with input: 4.518 kWh (QuantityType). Link: 'PV_H32_Energie_heute -> mqtt:topic:sbfspot_2100026732:EToday'
ant 11:55:37.213 | DEBUG | org.openhab.transform.basicprofiles.internal.profiles.StateFilterProfile | RHS: '50' is not compatible with LHS '4.518 kWh' (QuantityType)
ant 11:55:37.215 | DEBUG | org.openhab.transform.basicprofiles.internal.profiles.StateFilterProfile | Received state update from handler: 4.518 kWh, not forwarded to item

// after:
ant 11:57:36.172 | DEBUG | org.openhab.transform.basicprofiles.internal.profiles.StateFilterProfile | Evaluating Condition(''  LT '50 kWh' ) with input: 4.601 kWh (QuantityType). Link: 'PV_H32_Energie_heute -> mqtt:topic:sbfspot_2100026732:EToday'
ant 11:57:36.188 | DEBUG | org.openhab.transform.basicprofiles.internal.profiles.StateFilterProfile | Performing a comparison between input '16563600 Ws' (QuantityType) and value '180000000 Ws' (QuantityType)
ant 11:57:36.190 | DEBUG | org.openhab.transform.basicprofiles.internal.profiles.StateFilterProfile | Received state update from handler: 4.601 kWh, forwarded as 4.601 kWh

I think it would be good to add a breaking change warning and advice the users to check unit compatibility, WDYT?

@andrewfg
Copy link
Contributor Author

@florian-h05 thanks for reporting this. I suppose there are two ways forward..

  1. Let comparison targets without a unit be handled as if the decimal value would be in the unit of the target Item.
  2. Or as you suggest, mark it as a breaking change.

Probably 2. is the “correct” approach, but 1. is more lenient. Actually I thought that I had programmed it to do case 1. so I need to check. @jimtng any thoughts?

@jimtng
Copy link
Contributor

jimtng commented Feb 18, 2025

I thought that it required the unit from before this PR.

Prior to this PR, the unit of the linked item didn't matter. It only looked at the incoming value from the binding, which in this case would've been a QuantityType, and it would not allow comparison against a non QuantityType.

Since this PR isn't a patch, you could test it on 4.3.2 (old behaviour prior to this PR)

@andrewfg
Copy link
Contributor Author

andrewfg commented Feb 18, 2025

@jimtng thanks for the feedback. I think there are two topics..

  1. The question if the case of @florian-h05 is a breaking change: As I understand you, it was the same behavior prior to this PR so it's not a breaking change. Therefore no need for a warning in the release notes. Or?? However perhaps it means I should tweak the read me to make the use case clearer?

  2. The question if we should make another PR for the case where the incoming state is a QuantityType and the comparand has no unit; to assume that the unit is that of the target Item. I am not sure if this is a good idea. => WDYT?

@jimtng
Copy link
Contributor

jimtng commented Feb 18, 2025

  1. The question if the case of @florian-h05 is a breaking change: As I understand you, it was the same behavior prior to this PR so it's not a breaking change.

Correct, but I'm only 99% sure (don't remember exactly). We need feedback from @florian-h05 to see if he could test this on 4.3. I can also test tomorrow (it's very late here right now, I'm ready to hit the hay)

However perhaps it means I should tweak the read me to make the use case clearer?

What do you mean?

2. The question if we should make another PR for the case where the incoming state is a QuantityType and the comparand has no unit; to assume that the unit is that of the target Item. I am not sure if this is a good idea.

I don't think it's a good idea either. If my memory serves, this case was specifically not allowed since my first PR to this
See #17323
This specific test checked for it:

That was in August 2024.

@andrewfg
Copy link
Contributor Author

What do you mean?

I was thinking to make it clearer that the data type of the second comparand must match that of the first comparand in the chapter below.

https://www.openhab.org/addons/transformations/basicprofiles/#state-filter-conditions

@jimtng
Copy link
Contributor

jimtng commented Feb 19, 2025

I was thinking to make it clearer that the data type of the second comparand must match that of the first comparand in the chapter below.

I just had a look at the current doc (https://next.openhab.org) and I agree it isn't clear / emphasised that quantitytype data must be compared against a quantitytype constant / data, and comparison against plain number / non quantity will cause it to fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug An unexpected problem or unintended behavior of an add-on
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[basicprofiles] Calculations over series of QuantityType are based on the wrong Unit
4 participants